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Abstract 


This  paper  concerns  the  fundamental  problem  of  synchronising  communication 
between  distributed  processes  whose  speeds  (steps  per  real  time  unit)  vary 
dynamically.  Communication  must  be  established  in  matching  pairs,  which  are 
mutually  willing  to  communicate.  We  show  how  to  implement  a  distributed  local 
schedule*  to  find  these  pairs.  The  only  mean9  of  synchronisation  are  boolean 
•flag*  variables,  each  of  which  can  be  written  by  only  one  process  and  read  by 
at  most  one  other  process.  (Chared  variables  are  very  difficult  to  implement 
in  the  case  the  processes  are  running  in  different  processors  in  a  coiwnunicati.'.n 
network.)  No  global  bounds  in  the  speeds  of  processes  are  assumed.  Processes 
with  speed  zero  are  considered  dead.  However,  when  their  speed  is  nonzero  then 
they  execute  their  programs  correctly.  Dead  processes  do  not  harm  our  algorithms' 
performance  with  respect  to  pairs  of  other  running  processes.  When  the  rate  of 
change  cf  the  ratio  of  speeds  of  neighbour  processes  (i.e.  relative  acceleration) 
is  bounded,  then  any  two  of  those  processes  will  establish  communication  within 
a  constant  number  of  staps  of  the  slowest  process  with  high  likelihood »  So,  our 
implementation  has  the  property  of  achieving  relative  real  time  response.  We 
can  use  our  techniques  to  solve  other  problems  such  as  resource  allocation  and 
implementation  of  parallel  languages  tuch  as  CSP  and  Ada.  Note  that  we  do 
not  have  any  probability  assumptions  about  the  system  behavior,  although  our 
algorithms  use  the  technique  of  probabilistic  choice.^. 
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1.1  Abstract 

This  paper  concerns  the  fundamental  problem  of  synchronizing  communication 
between  distributed  processes  whose  speeds  (steps  per  real  time  unit)  vary 
dynamically.  Communication  must  be  established  in  matching  pairs,  which  are 
mutually  willing  to  communicate.  We  show  how  to  implement  a  distributed  local 
scheduler  to  find  these  pairs.  The  only  means  of  synchronization  are  boolean 
"flag"  variables,  each  cf  which  can  be  written  by  only  one  process  and  read  by 
at  most-  one  other  process.  (Shared  variables  are  very  difficult  to  implement 
in  the  case  the  processes  are  running  in  different  processors  in  a  communication 
network.)  No  global  bounds  in  the  speeds  of  processes  are  assumed.  Processes 
with  speed  zero  are  considered  dead.  However,  when  their  speed  is  nonzero  then 
they  execute  their  programs  correctly.  Dead  processes  do  not  harm  our  algorithms' 
performance  with  respect  to  pairs  of  other  running  processes.  When  the  rate  of 
change  of  the  ratio  cf  speeds  of  neighbour  processes  (i.e.  relative  acceleration) 
is  bounded,  then  any  two  of  these  processes  will  establish  communication  within 
a  constant  number  of  steps  cf  the  slowest  process  with  high  likelihood.  So,  our 
implementation  has  the  property  of  achieving  relative  real  time  response.  We 
can  use  our  techniques  to  solve  other  problems  such  as  resource  allocation  and 
implementation  of  parallel  languages  such  as  CSP  and  Ada.  Note  that  we  do 
not  have  any  probability  assumptions  about  the  system  behavior,  although  our 
algorithms  use  the  technique  of  probabilistic  choice. 

1.2  Introduction 

Recently,  [Reif,  Spirakis,  1981}  showed  how  to  achieve  real-time  response 
usinq  probaoi.listic  synchronization  techniques,  with  the  assumption  that  the 
speeds  of  all  processes  were  bounded  between  fixed  nonzero  bounds.  This  lead 
to  (see  appendices  I  and  II  of  [Reit,  Spirakis,  1981])  real  time  resource 
allocation  algorithms  and  real  time  implementation  of  message  passing  in  CSP. 

In  this  pa^er  we  assume  no  global  bounds  on  the  processors'  speeds.  They 
can  vary  dynamically  from  zero  to  an  upper  bound  which  may  be  different  for 
each  processor,  and  not  known  by  :he  other  processors.  We  allow  a  possibly 
infinite  number  of  processes,  so  that  there  may  not  be  a  global  upper  bound  on 
the  speeds .  Processes  may  die  (have  zero  speed)  but  when  they  have  nonzero 
speed  then  we  assume  they  execute  their  programs  correctly.  We  are  interested 
in  dip too  interprocess  communication  (rather  than  packet  switching)  which  is  of 
the  form  of  handshake  (rather  than  buffered),  as  in  Hoarefc  CSP.  [Hoare,  1978]. 

The  ess« ntial  technique  that  we  utilize  is  that  of  probabilistic  choice.  This 
technique,  introduced  tc  synchronization  problems  by  [Rabin,  1980],  [Lehmann  and 
Rabin,  1981]  and  [Francez,  Rodeh,  1980],  was  also  utilized  in  our  previous  work. 
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The  use  of  probabilistic  choice  in  the  algorithms  leads  to  considerable 
improvements  in  the  space  and  time  efficiency  [Riibin,  1980],  [Reif,  Spirakis, 

1981];  we  feel  that  this  may  be  because  complex  sequences  of  processes'  steps  * 

prohibiting  communication  have  vary  low  probability  of  occurrence.  We  also 

introduce  new  adaptive  teohniqu.ee;  The  processes  estimate  the  speeds  of 

neighbour  processes  and  select  them  to  communicate  with  probabilities  depending 

on  the  speeds,  penalizing  the  slowest  processes.  These  adaptive  techniques 

do  not  seem  to  have  ever  been  utilised  in  the  previous  synchronisation  literature. 

1.3  Relative  real  time  response 

If  processes  are  bounded  in  speed  then  it  is  natural  to  define  reel  time 
response  to  be  a  response  to  a  communication  request  that  uses  no  more  than  1 

constant  number  of  units  of  real  time.  This  measure  is  inapplicable  in  our  case  \ 

in  which  there  is  no  global  upper  bound  and  no  nonzero  lower  bound  on  speeds. 

Thus  wo  introduce  the  notion  of  relative  real  time  response  which  is  establish¬ 
ment  of  communication  between  any  pair  of  neighbouring  processes  within  constant  j 

number  of  local  rounds  (A  local  round  of  neighbour  processes,  i,j  is  the  minimum  | 

time  interval  which  contains  at  least  one  step  of  each  process  and  exactly  one  j 

stop  of  the  slowe" '  of  i,j).  We  achieve  this  by  our  probabilistic  algorithms  i 

with  some  probability  of  error  which  can  be  made  arbitrarily  low.  We  conjecture 
that  it  is  not  possible  to  achieve  relative  real  time  response  without  use  of 
randomization.  The  best  deterministic  symmetric  algorithms  which  attempt  to  f 

form  matchings  in  distributed  systems  have  a  relative  response  depending  linearly 
on  the  network's  diameter.  (Also,  [Arjomandi,  Fischer,  Lynch,  1981]  have  actually 
shown  that  some  synchronization  problems  which  are  global  (in  contrast  to  our 
problem)  cannot  be  done  in  real  time  and  require  time  proportional  to  the 
logarithm  of  the  total  number  cf  processors  in  the  network.  A  topical  situation 
where  this  could  occur  is  the  problem  of  detecting  connected  components  of  ; 

processes  whose  speeds  are  within  given  bounds  e.g.,  with  nonzero  values.) 

2.  The  model  VSDCS  (Variable  Speed  Distributed  Communication  System) . 

We  develop  here  a  theoretical  model  related  to.  but  more  general  than,  the 
DCS  (Distributed  Communication  System)  of  [Reif,  Spirakis,  1981].  We  will 
attempt  to  describe  our  model  in  words  and  avoid  mathematical  notation  in  this 
abstracted  draft.  A  detailed  description  of  the  fundamental  issues  can  be  found 
in  [Reif,  Spirakis,  1981]. 

We  assume  a  possibly  infinite  collection  of  processes  tt  =  {l,2,...}.  j 

Events  of  the  system  are  totally  ordered  on  the  real-time  line  [0,°°).  j 

Each  process  consists  oi  a  fixed  set  of  synchronous  parallel  subprocesses  j 

(i.e.  with  same  speeds)  where  we  distinguish  the  director  subprccess.  The  i 

director  wishes  at  various  times  to  communicate  with  directors  of  other  1 

processes  but  has  no  means  of  communication  except  via  the  communications  system.  j 

This  is  implemented  by  many  poller  subprocesses  (three  fcr  each  target  process) 
which  are  synchronous  with  themselves  and  the  director.  We  assume  a  fixed 
connections  graph  H  which  is  undirected  and  has  the  set  ir  as  its  vertex  set. 

An  edge  (i,j}  indicates  that  process  i  is  physically  able  to  communicate 
with  orocess  j  (but  not  necessarily  willing  to) .  H  is  assumed  to  have  finite 
valence.  We  also  assume  for  each  time  t  the  willingness  digraph  which 
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indicates  the  willingness  of  a  given  process  i  to  comunicate  with  a  given 
neighbour  j  at  a  given  time  t.  (We  indicate  it  by  the  edge  i  j  and 
say  i  is  a  willing  neighbour  of  j.). 

Note  that  i  only  if  U,j)  e  H 

The  edges  of  this  graph  are  stored  distributedly  so  that  the  edges  departing 
from  a  given  process  are  only  known  to  that  process.  We  assume  that  the  outdegree 
of  each  vertex  of  G  is  upper  bounded  by  a  fixed  constant  v.  For  each  t  _>  0, 
the  (possibly  infinite)  digraph  with  vertices  rr  and  directed  edges 

i  — j  denotes  which  processes  open  aorrmmioation  to  other  processes  at 

t 

time  t.  We  denote  i-*— ».j  if  both  and  j— Thus 

i  denotes  i,j  achieve  mutual  eormunioation  at  time  t.  Mt  is  t.n- 

digraph  that  implementations  of  distributed  synchronisation  achieve.  We  wish 
implementations  to  be  proper*  in  the  3ense  that 

(a)  i— - j  only  if  i  ■  ■  m  j  (neighbours  try  to  speak  only  if 
they  are  mutually  willing  to) 

(b)  -m - must  be  a  partial  matching. 

If  i  -m  then  not  j  « .«— AM— i  for 

any  j*  £  if  -  {j}.  (Nobody  is  allowed  to  speak  with  two  or  more 

neighbours  at  the  same  time . ) . 

Again,  we  assume  that  processes  can  suddenly  die  (i.e.  have  zero  speeds) 
but  when  they  are  awake  they  execute  their  programs  correctly.  We  furthermore 
assume  that  each  process  has  an  upper  bound  on  its  speed  which  may  be  different 
from  the  other  processes  and  not  known  to  them. 

We  define  the  relative  acceleration  bound  a  of  processes  i  and  j  to 
be  the  worst  case  absolute  rate  of  change  of  the  ratio  of  steps  of  the  two 
processes  per  step  of  any  of  the  two  processes.  The  correctness  of  our 
synchronization  algorithms  does  not  depend  on  whether  processes  are  acceleration 
bounded,  however  we  assume  fixed  acceleration  bound  O'  in  our  time  complexity 
analysis.  (i.e.  the  relative  acceleration  of  one  neighbour  with  respect  to 
another  is  bounded  by  a  constant  bound  a  or  can  be  if  the  process  dies) . 


2.2  Communication  commands  for  the  VS-OCS 

The  following  communication  primitives  can  bo  implemented  by  the  poller 
subsystem  and  executed,  by  the  directors  of  each  process:  (The  director  may  not 
get  an  immediate  answer  but  may  proceed  to  some  other  instruction  and  later  a 
time  slot  for  communication  will  be  arranged  by  the  poller) . 

ATTEMPT-COM^ (Pj)  :  indicates  that  the  director  of  i  wishes  to  communicate 

with  the  director  of  process  j. 

CANCEL-COM^ (p . )  :  indicates  that  the  director  of  i  wishes  no  longer  to 

communicate  with  p^. 


-  4  - 


The  precise  semantics  of  ATTEMPT -COM  and  CANCEL-COM  are  given  by  the 
relation  - - — ■»  c  t  x  it  (the  willingness  digraph,  defined  in  section  2.1). 

2.3  Complexity  of  VS-DCS 

We  assume  a  worst-case  oracle  tV  which  at  time  0  chooses  the  speeds  of 
all  the  processes  for  all  times.  otf  is  also  able  to  dynamically  change  the 
willingness  relation  1 " 1  ■— » (i.e.  dynamically  choose  one  of  ATTEMPT-COM, 
CANCEL-COM  for  the  directors  to  execute)  so  as  to  achieve  the  worst  case 
performance  of  the  implementation  of  VS-DCS 

We  say  ie  tame  for  t,  j  on  time  interval  A  if  the  pairs 
{ (i » j ) >  U  { (i ,k)  |  k  is  a  neighbour  of  i)  U  {(j,k)  |  k  is  a  neighbour  of  j} 
are  each  relative  acceleration  bounded  by  a  on  the  time  interval  A. 

(We  assume  here  a  global  constant  a) . 

For  every  e  on  (0,1)  let  the  £-error  response  Sic)  be  an  integer  >  0 
such  that  for  every  pair  of  neighbours  i,j  and  each  tit  interval  A  and  for 
every  oracle  oaf  which  is  tame  for  i,j  on  A,  if  i  ^  ^  >  j  and  the 

number  of  steps  of  the  slowest  of  i,j  within  A  is  >_  S(e)  then  there  exists 
a  subirterval  A'  c  A  such  that  i  —  j  with  probability  >  1-  £. 

Intuitively  1-e  gives  a  lower  bound  in  the  probability  of  establishing 
communication  in  the  case  process  i  issues  an  ATTEMPT-COM  (p J  at  the 
beginning  of  A,  and  after  S(C)  steps  it  calls  CANCEL-CCM  (p.),  (Note  that 
we  presume  here  that  i  and  j  and  their  neighbours  have  relative 
acceleration  bound  a  during  the  interval  A,  at  other  times  this  acceleration 
bound  may  oe  violated,  and  furthermore  the  acceleration  bound  a  need  not  hold 
for  other  processes  even  during  the  interval  A) . 

We  consider  an  implementation  of  VS-DCS  to  be  relative  y'eal  time  if  for 
all  constants  e  on  (0,1),  the  relative  e-error  response  S(e)  is  independent 
of  any  global  measure  of  the  willingness  digraph  (such  as  jirj  or  any 

function  of  it)  and  independent  of  any  local  measure  of  except  the 

constant  maximum  valence  v  of  the  vertices  of  Gfc.  (3(e)  may  depend  on 
the  bound  a  on  the  relative  acceleration) .  Note  that  relative  real  time 
response  does  not  imply  that  communication  is  guaranteed  within  any  time 
interval  but  instead  it  is  guaranteed  within  a  bounded  number  of  steps  of  the 
processes  with  high  likelihood  (this  is  because  processes  can  slow  down 
arbitrarily)  .  In  this  paper  we  show  how  to  .implement  the  VS-DCS  so  that  relative 
real  time  response  is  achieved. 
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3.  Applications  of  VS-DCS  { 

f 

i 

■f 

The  primitives  ATTEMPT -COM,  CANCEL-COM  of  VSDCS  are  powerful  enough  r 

to  supply  real  tiros  implementations  of  synchronisation  constructs  of  high-level 
parallel  languages  like  CS?  and  Ada.  j: 

{ 

3.1  Real  time  resource  granting  systems  with  process  failures 

j . 

Previously,  in  fReif,  Spirakis, 1981]  we  utilised  the  more  restricted  DCS  Pl¬ 
ays  tem  (which  does  not  allow  process  failures)  to  implement  a  real  time  resource  !  ;.y 

granting  system.  In  this  paper,  we  can  cope  with  sudden  process  failures  (sero 
speeds).  In  this  case,  the  process  governing  a  resource  will  attempt  to  communi¬ 
cate  for  as2 (e) /2  of  its  steps  with  a  process  granted  the  resource.  If 
that  process  does  not  respond,  the  resource  governing  process  may  reclaim  the 
resource.  If  a  resource  allocator  dies,  then  other  processes  cam  play  its  role. 


3.2  Relative  real  time  implementation  of  CSP  and  Ada's  synchronisation  constructs. 

i 

In  a  typical  stage  during  execution,  the  processes  comprising  a  CSP  program 
may  be  divided  into  two  classes:  those  busy  with  local  computations  and  those 
waiting  for  a  partner  to  communicate  with.  A  distributed  guard  scheduler  can  be 
implemented  by  using  the  poller  subprocesses  of  the  relative  real  time  VS-DCS 
system. 

Also,  in  Ada,  two-way  communication  between  pairs  of  tasks  is  allowed  in 
synchronized  time  instances  called  rendezvous.  An  aooept  statement  of  the  form  \ 

accept  f(-)  appearing  in  task  indicates  that  T^  is  willing  to  rendezvous 

at  f  with  any  task  of  similar  argument  type.  The  task  ’i  2  may  execute  a  call 

statement  of  the  form  f(-)  indicating  that  is  willing  to  rendezvous  with 

T.  at  the  accept  statement  containing  f.  Ada  also  allows  for  selective 
accept  statements  containing  multiple  accept  statements,  one  of  which  must  be 
nondeterministically  chosen  to  execute.  (This  is  similar  to  the  select 
statement  of  CSP) . 

Ada's  tasks  may  be  implemented  by  processes  whose  speeds  vary  dynamically. 

(Processes  may  even  fail  for  various  time  intervals.)  The  key  implementation 

problem  is  to  synchronize  task  rendezvous  within  relative  real  time,  in  spite  of 

the  dynamic  speed  variations.  These  processes  may  be  connected  within  a 

distributed  network  whose  transmission  channels  may  also  have  variable  speeds  or 

fail.  Unreliable  transmission  channels  can  be  viewed  as  processes  which  are  j 

connected  with  the  processes  of  the  network  via  reliable  communication  channals.  j 

We  assume  that  it  is  possible  to  analyze  (perhaps  by  data  flow  analysis) 
an  Ada  program  to  determine  an  indirected  (possibly  infinite)  connections  graph 
whose  nodes  are  all  the  tasks  possibly  created  by  the  Ada  program  and  edges  are  h 

h 
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the  possible  task  communication  pairs.  Since  an  actual  implementation  will 
have  in  its  hands  at  any  time  only  a  finite  set  of  processes  we  assume  that 
only  the  currently  active  tasks  have  an  associated  implementing  process  and 
that  a  call  to  Ada's  initiate  statement  devotes  a  currently  free  process  to 
a  given  newly  created  task.  An  abort  statement  garbage  collects  the  implementing 
process  from  the  deleted  task  and  places  it  beck  to  the  free  list  of  processes. 
These  implementation  techniques  were  developed  by  [Denis  and  Mi Bunas,  1974]  for 
real  time  implementation  of  data  flow  machines. 

The  synchronization  facilities  of  the  VS-DCS  system  provide  a  real-time 
implementation  of  the  accept  and  call  statement#.  A  version  of  the  active 
statement  can  be  implemented  so  that  deleted  tasks  can  be  detected  by  their 
neighbours  in  real  time  with  some  (arbitrarily  small)  error  probability.  Finally, 
the  eyrnetr y  and  locality  of  the  VS-DCS  implementation  (due  to  its  probabilistic 
nature;  may  help  in  eliminating  the  tradeoff  between  generality  of  expression 
and  ease  of  implementation  in  Ada. 

The  probabilistic  fairness  guaranteed  by  the  algorithms  of  the  pollers 
eliminates  the  danger  of  bottlenecks  which  could  be  created  if  conventional 
techniques  were  used  (a  r.ew  task  which  centralizes  requests  and  keeps  track  of 
busy  server  tasks  is  one  of  the  conventional  proposed  solutions) .  Most  of  the 
problems  which  VS-DCS  could  cure  are  discussed  in  [Mahjoub,  1981],  [Frances, 

Rodeh,  1980].  A  probabilistic  solution  to  some  of  the  discussed  problems  was 
given  also  in  [Frances, Rodeh,  1980]  but  no  discussion  about  real-time  properties 
was  done  and  neither  the  problem  of  speed  variations  and  dying  processes  was 
addressed. 

4 .  Relative  Real  Time  Implementation  of  VS-DCS 
4.1  Intuitive  description  of  the  algorithm. 

We  utilize  3v  +  1  synchronized  parallel  processes  to  implement  the  poller 
subprocess  for  each  process  i .  These  are  the  communicators 

Pi»  pi#-"-*  pi  *  the  speed  estimators  e. , . . . ,  e1  and  the  judge  subprocess  of 

l  iv  i 

process  i.  Each  pair  of  the  communicators  p.,  p.  „  (with  k'mod  v  =  k"  mod  v  *  k) 

^  ^ 

is  devoted  to  communication  with  a  specific  neighbour  (the  k  neighbour) .  Each 
estimator  is  used  to  continuously  update  an  estimation  of  the  speed  of  a 
particular  neighbour  process.  The  judge  lias  the  task  to  select  under  certain 
conditions  one  communicator  and  to  give  to  him  the  right  to  open  the  communica¬ 
tions  channel  of  node  i  to  its  corresponding  neighbour.  We  frequently  use  the 
technique  of  handshake  by  which  we  mean  that  each  subprocess  modifies  a  flag 
variable  observed  by  the  corresponding  neighbour  subprocess.  Process  contention 
between  synchronized  processes  is  easy  to  implement  (we  can  allow  each  to  take  a 
separate  step  in  a  small  round) . 

Our  algorithm  for  the  k*^  communicator  subprocess  (1  £  k  <_  2v)  of  the 
poller  of  process  i  proceeds  as  follows: 
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Let  k‘  «  k  modv.  At  every  time  t  >0,  E^l),...,  e^d^)  is  the  list 
of  targets  of  edges  of  departing  from  i  e  it,  and  is  the  current  number 
of  the  targets  (D^  £  v) .  Those  variables  are  dynamically  set  by  the  oracle  .*/ 
and  they  are  the  neighbours  to  which  process  i  is  willing  to  open  communica¬ 
tion  at  time  t.  The  subprocess  p*  deals  with  the  E^(k')'  neighbour.  If 
k  <  v,  then  p£  is  an  aaker  subprocess,  else  it  is  a  responder  process,  p*  must 
first  handshake  with  the  corresponding  subprocess  of  process  E^(k*)  to  which 
node  i  wishes  to  communicate.  We  need  two  handshake  subprocesses  (ask,  respond 
respectively)  per  neighbour  because  of  a  certain  asymmetry  in  the  handshake  (some 
has  first  to  modify  a  flag) .  in  particular  the  aaker  procedure  initiates  the 
handshake  and  the  responder  answers  to  it. 

Next  we  wish  to  find  a  time  slot  in  which  the  two  neighbours  may  communicate. 
Because  there  may  be  contention  among  other  processes  j  which  also  wish  to 
communicate  with  i  (and  consequently,  other  makers  or  responders  of  node  i 
also  will  handshake)  we  must  resolve  the  contention  by  a  fair  judge.  To  do 
this,  we  add  the  process  p^  to  a  queue  and  an  additional  synchronous 
subprocess  of  poller  i,  the  judge  takes  a  random  process  from  this  queue  and 
allocates  time  slots  for  communication  attempts.  To  ensure  that  slower 
neighbours  do  not  utilize  any  more  total  time  on  the  average  than  faster 
neighbours  during  communication  attempts,  we  weigh  the  probabilities  of  sub¬ 
processes  to  be  chosen  from  the  queue  by  the  factor 


where  A  (m  -  1,...,  v)  is  the  current  estimation  of  the  steps  of  process  i 

per  step  of  process  m,  supplied  by  the  estimator  e*. 

m 

This  has  the  effect  that  each  oubprocess  in  the  queue  attempts  to  communicate 
on  the  average  l/2v  of  the  total  time.  (See  the  analysis  for  a  proof  of  that) . 
If  a  process  is  chosen  by  the  judge  but  the  communication  is  not  established,  the 
algorithm  requires  that  subprocess  to  initiate  another  handshake  with  its  partner 
(to  check  if  they  are  still  mutually  willing  to  communicate  and  to  synchronize 
steps).  Then,  it  is  again  added  to  the  queue  to  be  given  another  chance  to 
establish  communication.  This  process  proceeds  until  either  the  director  of  i 
withdraws  its  willingness  to  communicate  with  E^(k')  or  until  it  establishes 
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communication.  Note  that  the  slots  allocated  by  the  judge  to  each  selected 
communicator,  take  into  account  the  current  speed  ratio  of  the  node  i  and  its 
neighbour  corresponding  to  that  communicator,  adjusted  by  a  factor  related  to 
worst-case  acceleration,  to  give  tne  opportunity  of  at  least  one  step  overlap  In 
time  of  process  i  and  its  neighbour,  if  their  corresponding  channels  are  both  open. 

We  introduce  random  waits  which  help  subprocess  p*  to  eliminate  the 

possibility  of  schedules  set-up  by  the  orecle  V  to  have  always  a  particular 
subprocess  arrive  first  in  the  queue  and  win  the  contest.  This  possibility  is 
eliminated  since  the  oracle  sets  the  speeds  at  time  0  and  cannot  affect  the 
random  choices  done  by  the  processes. 

Also,  our  algorithms  assume  each  process  has  a  perfect  random  bit 
generator,  independent  of  the  random  bits  generated  by  other  processes. 

Note  that  we  trade  computation  effort  (parallelism)  in  a  node  to  achieve 
reliable  communication.  This  parallelism  is  limited  because  of  the  bounded 
valence  v  of  the  graph  Gfc.  We  can  always  simulate  these  synchronous 
parallel  subprocesses  in  a  node  i  by  a  single  processor,  using  round  robin 
techniques.  This  will  reduce  the  effective  speed  of  each  subprocess  by  only 
a  factor  of  3v  +  1. 


4.2  The  programs  of  the  pollers 


For  each  poller  i  e  it  ,  we  assume 


Synchronous  sjbprocesses 
i  i  i 

i 


•v+1 


9  •  •  •  9 


v,,,#  % 


and  the  judge. 


Bach  of  the  communicators 

process 


(askers) 

(responders) 

(estimators) 

Pk  executes  the  following  program 


WHILE 


00 


locals  A., 

-  ik 

true  DO 

LO:  IF  D.  >  k  THEN 
BEGIN  . 

Llj  Choose  W  at  random  from  [0,4(2v+D (2a+l) ] 
do  W  noops 

if  k<v  then  ASK(Ei  (k) )  else  RESPOND^  (k) ) 
add  k  to  Q 

WHILE  marriage  (k) =0  DO  noop 

X -*-EST;»BLI3H-C0M  (Ei(k),4(2  Ct  +  1)  •  Aik) 
marriage , (k)  0 

IF  X  THEN  GO  TO  Ll  ELSE  GO  TO  LO 

END 


\ 
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The  speed  estimator  e^j 
i 

process  e^ 

DO  FOREVER 


set 

F 

ik 

to  1 . 

wait 

until 

Pki 

is 

set 

zero 

Fik 

;  s 

•*-  CURSTEP 

wait 

until 

Fki 

is 

zeroed 

A 

CURSTEP 

-s 

ik 

2 

OD 

Noto  that  Fik  is  a  flag  set  by  i,  read  by  k. 

The  special  register  CURSTEP  gives  the  current  step  of  process  i.  We 
assume  that  a  step  consists  of  an  elementary  statement  of  the  progra  «  re^'s 
execution  assures  that  A^  is  (within  a  factor  of  2)  the  actual  speed  ratio 
of  processes  i  and  k,  since  from  step  A  to  step  B  the  fastest  of  the 
partners  does  CURSTEP-s  steps  and  the  slowest  does  2  steps 


process  judge ^ 

WHILE  true  DO 

IF  g  /  0  THEN 


BEGIN 

choose  and  delete  a  random  element  k  of  the  queue 

Q  with  probability  *^ik 

I  1/A.  . 

j  ^ 


marriage  ^(kH —  1 

WHILE  marriage ^(k)  =1  DO  noop 
END 

END 

OD 

Note ;  The  assignment  of  the  marriage^  variable  by  the  judge  allows  p^  to 
attempt  a  communication  with  the  corresponding  subprocess  of  che  neighbour 
E, (k‘)  . 


jf 

f 


i 
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The  following  are  the  low  level  synchronization  procedure*:  used  by  the 
poller  programs : 

procedure  aski  (target) 

BEGIN 


0.  1 : 

“i, target 

WHILE  A.  .  ,  =  0 

-  trget,i 

DO 

noop 

Q.  *♦"  0: 

i, target 

WHILE  „  .  »  1 

-  target ,i 

DO 

noop 

END 


Note:  The  set  of  the  flag  QJ#,  means  that  i  asks  the  target.  If 

-  i  target 

the  target  detects  Q.  .  .  ■=  1  ..  . .  _  . 

l, target  then  it  answers  positively  by  setting 

A  =  l .  Both  partners  reset  these  flags  to  O  at  the  end  of  procedures  ask 

target,i 
and  respond* 

procedure  respond^  (asker) 

BEGIN 

LOOP  UNTIL  Q  ,  .  =  1 

BEGIN 

Ai, asker  **  ^ ' 

55^  Qasker,i  =  1  5°  no° p 

Ai, asker  0; 

END 


END 


We  finally  present  the  code  for  the  procedure  ESTABLISH-COM^ (target, l) .  During 
its  execution  node  i  opens  its  channel  to  node  target.  A  simple  protocol 
(symmetric  handshake)  is  then  attempted  to  see  if  the  neighbour  responded  to 
that  communication  attempt.  If  the  protocol  succeeds  then  node  i  is  3ure  that 
node  target  also  opened  its  channel  and  communication  took  place.  Else,  node  i 
knows  that  the  attempt  failed. 

procedure  ESTABLISH-COM^  (target,?-) 


BEGIN 

open  channel 


i, target' 


So  4-  CURSTEP 
b  4-  1 


COM. 

l, target 


1 


WHILE  (COM.  .  .  =  0)  or  (b  *  1)  DO 
-  target,i  — 

BEGIN 

IF  CURSTEP-So  >  i  l.iEW  b  4-  0 
END 


OD 


..  .  - 
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IF  (COM,  „  .  »  \)  AND  (b  -  ?,)  THEN  success— -1 
—  target,!  -  "  ~  • 


b  •* — 0 


COM.  „  ♦ 

i, target 


close  channel.  ^ 

i, target 

return  (success) 


ELSE  success 


Note:  COM,  is  a  flag  of  node  i  and  COM,.  .  .  is  a  flag  of  node 

-  i, target  target,  i 

target,  open  channel.  corresponds  to  the  appearance  of  i — ^ - ►target 

at  the  time  of  its  execution.  Also,  l  is  the  maximum  number  of  steps  we  are 
allowed  to  keep  channel  open  before  we  fail. 

5.  Correctness  properties  of  our  proposed  implementation  and  time  analysis. 

Lemma  5.1  A  matching  (with  respect  to  the  relation  - ►  )  is  guaranteed 

by  the  implementation. 

Proof 

In  any  time  instant,  only  one  of  the  subprocesses  of  any  poller  can  have 
the  marriage  variable  set  and  its  channel  open.  So,  the  relation - 


is  one-one  which  means  that 


can  not  be  more  than  a  matching. 


Lemma  5.2  Death  of  a  process  does  not  affect  the  communication  of  other 


processes. 


Proof 


Death  of  process  "target"  at  any  time  will  only  cause  blocking  of  only  one 
subprocess  (P^arget*  per  ne^9ht,our  *  of  target.  This  does  not  disrupt  the 
other  subprocesses  of  the  neighbours. 

Lemma  5.3  Suppose  that  i,j  start  to  be  mutually  willing  to  communicate  at 


time  t  and  continue  to  be  willing  for  5  local  rounds.  Then  all 
four  subprocesse3  p^  ,  p^-  and  p?  ,  p?  (with  j1  modv  =  jr  modv  =  j 

-'l  -*2  *i  X2  1  1 

and  i^  modv  =  i 2  modv  =  i)  will  arrive  in  the  queues  of  i  and  3 
in  a  constant  number  (5)  local  rounds. 


Proof 


Note  that  at  each  time  the  slower  of  i,j  will  do  only  one  step  in  the 
busy  waits  of  procedures  ask  or  respond.  The  result  follows  simply  by  counting 
the  steps  to  be  executed  in  each  of  the  procedures .  o 

Let  A^  be  the  current  estimation  (within  a  factor  of  two)  of  the  ratio 
of  steps  of  i  per  step  of  j  (estimated  by  i) . 


. .  ,  Tif"  XW'TT’fniiMwwnr  .■wumwmi »..««»  ..... 
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Definition  5.3 


Let  h  =  I 


k«l  ik 


Definition  5,4  Let  be  the  ratio  l/<h*A^) 

In  the  following  we  assume  that  the  oracle  u#  is  tame  with  respect  to 
processes  i,j  in  the  time  interval  they  attempt  communication. 

Definition  5.3  Let  be  the  average  number  of  steps  that  p*  does 

before  it  is  selected  to  attempt  communication,  measured  from  the  time  it  enters 
the  queue. 

Lemma  5.4  £  8v»  (2ct+l)  *A^ ^  where  A^  is  the  most  current  estimation 

Proof 

Note  that  the  probability  to  be  chosen  follows  a  geometric  density 
(1  -P.  . )k  P. .  where  h  *  the  number  of  selections  done  before  p*  is  selected. 

J  i  J  J 

Each  time  p_.  is  not  chosen,  it  waits  in  the  queue  for  an  average  time  bounded 

above  by  2v 

1  c*Aik  Pik 
k=l  K 


where  c  =  4(2a+l). 


m  /  \  2V 

^ ,  £  I  prob/h  unfortunate  ij*  h  c* 

^  h-1  Selections  for  Pj/  k=l 


Aik  Pik 


2v  2v 

-  *  I  c*A..  p..  =hA.  .  7.  —  =  2vcA.  . 

p,  .  .  ,  ik  ik  i]  ,  ,  h  i] 

ij  k=l  J  k=l  J 


Theorem  5 . 1  Each  subprocess  expects  to  get  the  channel  >  l/2v  off  the  time. 


Proof 


The  average  number  of  steps  p,  get®  the  channel  is 


In  the  worst  case  of  process  i  being  the  fastest  and  all  neighbours 
slowing  down  with  the  same  worst  case  acceleration  <x ,  is  the  same  for  all 

processes  in  the  queue.  The  worst  contention  happens  when  all  2v  subprocesses 

are  there.  Hence,  in  the  worst  case,  if  T  is  a  time  interval  and  T  is  the 

i  * 

subxnterval  of  T  in  which  p  has  the  channel,  then 


as  the 


""""  \WT  J  -  2vs  2v 

2  1/Aik 

Note  that  this  justifies  the  use  of  the  estimate  — ■  -r-;v —  as  the 

i  Z 1/Aii 

probability  to  select  subprocess  p *  from  the  queue,  j  . 

k  ^ 

In  the  following  we  assume  1  £  i,k*  £  v  and  k  *  k'  +  v.  Thus  p.  ,  is  the 

k  * 

asker  and  is  the  responder. 
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Lemma  5.5  The  probability  of  instantaneous  overlap  of  subprocossas  p,  and 

-  k*  /l  \2  k 

in  their  channels  is  >_  I  — v  j 

Proof 


By  theorem  5.3  and  by  the  fact  that  the  random  waits,  done  by  the  subprocesses 
before  each  return  to  the  queue,  cause  the  relative  position  of  the  time  intervals 
during  which  channels  are  open  to  be  random,  not  affected  by  the  oracle  ot  . 

(Note  that  it  is  essential  here  that,  the  waits  are  uniformly  distributed  in  the 
interval  whose  length  is  the  mean  number  of  local  rounds  attempt  communication  and 
given  by  Theorem  5.3).  0 

Definition  5,6  Let  euaoess  in  aormuniaation  be  an  overlap  of  open  channels  for 
at  least  one  step  of  both  processes  i,k*. 

Definition  5.7  A  phase  of  subproces8  is  a  random  wait,  a  handshake  with 

k  * 

p^,  a  wait  in  queue  and  a  communication  attempt. 


Theorem  5.2 


The  probability  of  success  in  communication  in  a  phase  of 
?  A  \2  1 


subprocess  p  is 
Proof 


When  the  subprocess  p^  opens  its  channel,  the  number  of  steps  done  from 


the  time  of  the  last  estimation  of  A. 


is  at  most  2A.,  and  hence,  the  new 
lk 


speed  ratio  can  be  (2a+i)  A^  in  the  worst  case,  (in  which,  process  i  is 

the  fastest  and  process  k •  slows  down  continuously  with  the  maximum  acceleration, 

so  that  process  i  does  more  and  more  steps  per  step  of  ic • ) .  In  this  case,  a 

k 

communication  attempt  of  4(2a+l)A..  time  slots  guarantees  that  p.  will  do  at 

•  X 

least  2  steps  during  the  time  p.  has  its  channel  open.  Because  of  the  random 

K  i 

relative  position  of  these  steps  with  respect  to  p  's  steps  (due  to  random  waits), 

*  2-1  1 

Prob  (length  of  overlap  is  >  1  step  given  that  there  is  an  overlap)  >  — —  «  r? 

—  «—  4  4* 

Hence 

Prob  (there  is  an  overlap  and  its  length  is  >_  1  step  of  both  processes) 
l  /i  \2 


by  Lemma  5 . 3 


Definition  5.8 


Let  Y  ■ 
min 


Definition  5.9  Let  q. 


.  i  (I -  )2 
2vU  J  * 

( be  the  p 


be  the  probability  that  it  takes  exactly  h 


phases  for  poller  subprocess  p  to  •ommunicate  with  p, 

X  X 


Lemma  5.10 


For  ary  oracle  (hfaj  <  (1-Ym±n)h“1 
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Proof 


It  sufficies  to  observe  that  the  process  of  p*  be  answered  by  p*'  is 
a  geometric  stochastic  process  with  success  probability  bounded  by  fY^^l]. 
By  using  the  above  lemma  and  known  expressions  for  the  mean  and  the  tail  of  a 


geometric  we  get 
Lemma  5.7  mean  (h)  < 


Lemma  5.8 


<W 


V  £  (  0  <  e  <  1  ,  Prob  {h  >  h  (E>)  <  E 

max  — 


where 


hM,(«) 

max 


*°»  'W1 


(l-w 


Note  that,  in  the  worst  case  relation  of  speeds  of  processes  i,k  ,  the  total 

length  of  a  phase  of  subprocess  p1  is  the  number  of  local  rounds  in  the  random 

k 

wait  plus  the  number  of  local  rounds  up  to  the  end  of  the  communi cation  attempt, 
which  is  S (2v+i) (2a+l)  (by  the  algorithms  and  by  Lemma  5.4  and  Theorem  5.1.) 
Hence  we  get 

Theorem  5.3  For  the  worst  case  oracle  ,  the  mean  number  M  of  local  rounds 

to  achieve  communication  is  . 

M  <,  64v  8  (2v+l)  (2ct+l) 

and  the  e-error  response  S(e)  of  the  presented  implementation  of  vs-DCS  is 


S(e)  < 


8 (2v+  l  (2a+l)  •  h  (e) 
max 


Proof 


=  O(vct) 


and  S(e) 


>£og  ( 


f»  •«) 


By  previous  remark  and  the  fact  that 


h  (e) 
max 


log (e/8v  ) 
logTl  -  — . 


16v2Jlog(|) 


Conclusion 

Since  we  have  assumed  global  parameters  a  and  v  to  be  constant,  by 
Theorem  5.3  our  system  has  relative  real  time  response.  Our  restrictions  on 
processors  rates  are  much  less  than  in  our  previous  paper  [Reif,  Spirakis,  1981]. 
Furthermore,  our  programs  seem  much  more  modular  and  simple  in  design,  although 
we  have  utilized  new  adaptive  techniques  to  deal  with  arbitrary  speed  variability. 
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